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Abstract 

In  this  paper,  we  trace  ou£s>the  solution  of  large  optimization 
problems  for  regional  planning^by  a decomposition  program  that  is  based 
on  the  use  of  standard  LP  programs.  The  underlying  method  is  the  decom- 
position principle  of  Dantzig  and  Wolfe. 

The  concept  is  tested  by  solving  an  optimization  problem  with 
about  1250  columns  and  900  rows.  Furthermore,  it  is  investigated  to 
what  extent  the  efficiency  of  the  solution  procedure  can  be  influenced 
by  an  appropriate  choice  of  starting  solutions  or  specific  matrix 
divisions. 


1.  Introduction 

The  programming  formulation  of  a large  spatial  equilibrium  model 
of  the  agricultural  sector  of  Germany,  which  is  in  preparation,  results  in 
a very  large  linear  programming  problem  with  a block-angular  structure  in 
the  problem  matrix.  For  solving  it,  existing  standard  computer  programs 
for  linear  programming  problems  (standard  LP  programs)  provide  too  little 
capacity. 

In  this  paper,  we  trace  out  the  solution  of  such  problems  by  a 
decomposition  program  that  is  based  on  the  use  of  Standard-LP-Programs. 

The  underlying  method  is  the  decomposition  principle  of  Dantzig  and 
Wolfe  [ 1].  The  concept  is  tested  by  solving  a similar  but  smaller  prob- 
lem with  about  1250  columns  and  900  rows.  Furthermore,  it  is  investigated 
to  what  extent  the  efficiency  of  the  solution  procedure  can  be  influenced 
by  the  choice  of  starting  solutions  or  specific  matrix  divisions. 

Some  of  the  results  are  of  general  interest  for  the  solution  of 
linear  optimization  problems  with  a similar  matrix  structure.  They  are 
based  on  experiments  that  have  been  partly  reported  in  [ 5]  and  [ 6]  and 
that  are  outlined  in  detail  in  section  4 of  this  paper. 

2.  Background  information 

Since  1970,  a large  spatial  equilibrium  model  of  the  agricultural 
sector  of  Germany  is  in  preparation.  The  first  (completed)  version  of  the 
model  was  designed  as  a linear  optimization  problem  with  about  8000  con- 
straints. While  this  problem  could  still  be  solved  by  means  of  standard 
LP  programs,  the  formulation  of  a more  disaggregated  version  and/or  the 
incorporation  of  other  "Common  Market"  countries  (suggestions  which  are 
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under  consideration)  would  result  in  a model  too  large  for  the  available 
programs  (see  appendix  1). 


However,  the  special  (block-angular)  structure  of  the  problem 
matrix  that  was  due  to  the  distinction  of  a limited  number  of  "regions" 
as  aggregated  economic  units  of  production  and/or  consumption^  would 
allow  the  solution  of  the  problem  by  means  of  a computer  program  that  is 
based  on  an  appropriate  decomposition  method  (see,  for  example,  Geoffrion 
[ 2]). 

However,  despite  the  fact  that  decomposition  methods  are  aimed 
at  solving  large-scale  optimization  problems,  efficient  programs  for  the 
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solution  of  really  large  problems  are  not  commonly  available.  Of  the 
standard  LP  programs, only  "LP  600/6000"  and  "OPTIMA" 

offer  (as  an  alternative)  a decomposition  procedure  that  could  be  used 
for  solving  large-scale  optimization  problems.  Unfortunately,  these  pro- 
grams are  designed  for  the  use  of  specific  computers. only. 

As  the  development  of  a specific  decomposition  program  was  not 
possible,  it  was  suggested  to  base  the  solution  of  the  optimization 
problem  on  a decomposition  approach  that  allowed  the  use  of  one  of  the 
highly  efficient  standard  LP  programs.  The  approach  that  has  been  devel- 
oped is,  in  principle,  based  on  the  decomposition  method  of  Dantzlg 
and  Wolfe.  It  is  outlined  in  more  detail  in  the  following  chapter. 

The  efficiency  of  the  concept  has  been  tested  in  several  experi- 
ments with  a similar  but  smaller  optimization  problem.  The  organization 
of  the  experiments  and  the  results  of  the  computations  are  discussed  in 
chapters  . 4 and  5 and  in  appendices  2,  3 and  4. 


1)  For  a detailed  discussion  of  such  models  see  [ 7]. 


During  the  following  discussion, we  assume  a familiarity  with  the 
decomposition  principle.  Its  application  requires  the  division  of  the  op- 
timization problem  into  "subproblems"  and  a "master  problem". 


The  matrix  of  each  subproblem  is  formed  by  one  or  several  of  the 
diagonal  "submatrlces"  of  the  block-angular  problem  matrix. 

The  matrix  of  the  master  problem  includes  the  common  constraints 
of  the  problem  matrix  and,  in  addition,  might  also  include  one  or  several 
of  the  diagonal  submatrices. 

^ In  an  iterative  solution  procedure,  the  decomposition  principle 

generates  columns  for  the  master  problem  from  solutions  of  the  subproblems 
and  objective  functions  for  the  subproblems  from  solutions  of  the  master 
problem. 
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3.  Computational  concept 

In  an  optimization  process  based  on  the  decomposition  method  of 
Dantzig  and  Wolfe  [ 1],  standard  LP  programs  can  be  used  for  solving  the 
subproblems  and  the  master  problem  in  each  of  the  cycles  of  the  Iterative 
solution  procedure.  In  this  case,  only  computer  programs  have  to  be  devel- 
oped that  transform  the  solutions  of  the  subproblems  and  the  master  prob- 
lems according  to  the  requirements  of  the  decomposition  method. 

Like  most  of  the  available  standard  LP  programs  for  large  problems 
(see  appendix  1),  the  standard  LP  program  that  was  available  for  the  experi- 
ments (program  XDLA  of  ICL)  could  not  be  used  as  a subprogram  in  computer 
programs  which  were  written  in  a user-oriented  programming  language  like 
ALGOL  or  FORTRAN.  j 

The  experiments  were,  therefore, based  on  a cycle  of  programs  which 
included : 

1)  standard  LP  program  for  solving  the  subproblems; 

2)  FORTRAN  program  for  generating  columns  for  the  master  problem; 

3)  standard  LP  program  for  solving  the  master  problem; 

4)  FORTRAN  program  for  computing  revised  objective  functions  for  the 
subproblems. 

During  the  solution  procedure,  this  cycle  had  to  be  activated  repeat- 
edly. Each  repetition  corresponded  to  one  cycle  of  the  solution  procedure 
(i.e.,  one  Iteration  of  the  decomposition  process). 

The  optimization  approach  differed  from  the  original  presentation 
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by  Dantzig  and  Wolfe  Insofar  as  the  following  suggestions  for  shortening 
the  solution  procedure  could  be  realized  within  the  outlined  concept: 


subproblem  solutions  from  cycle 


AO- 
35- 
JO- 
25- 
20- 
15- 
10- 
5 • 


-i 1 1 - i - i 1 1 

5 10  IS  20  25  30  35  AO 

cycle  of  the  solution  procedure 


Exhibit  1:  Solutions  of  subproblems  that  were  included  in  the 

optimal  basis  solution  of  the  master  problem  in  dif- 
ferent cycles  (example:  experiment  I). 


simplex 


Exhibit  2:  Number  of  simplex  iterations  that  were  necessary  for 

solving  the  subproblems  in  different  cycles. (In  experiment 
VII, the  optimizations  were  based  on  optimal  basis  solutions 
from  preceding  cycles;  in  experiment  I, on  the  optimal  basis 
solutions  from  the  first  cycle.) 
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1)  In  any  cycle  of  the  solution  procedure, the  master  problem  included 


all  columns  that  had  been  generated  from  solutions  of  the  subproblems 
in  preceding  cycles  (see  Lasdon  [ 3])*  However,  the  experiments 
showed  that  the  optimal  basis  solution  of  a master  problem  usually 
included  only  variables  which  referred  to  solutions  of  subproblems 
that  (a)  were  obtained  in  some  of  the  latest  cycles  and  (b)  were  in- 
cluded in  the  optimal  basis  solutions  of  preceding  cycles,  i.e.,  in 
cases  of  limited  storage  capacity,  columns  that  had  been  generated 
during  preceding  cycles  could  have  been  deleted  if  the  corresponding 
variables  were  no  longer  realized  in  the  optimal  basis  solution  of 
a master  problem  (see  exhibit  1 ) . 

2)  The  optimization  of  subproblems  and  master  problem  in  each  cycle  was 
based  on  the  optimal  basis  solutions  that  had  been  obtained  in  the 
preceding  cycle.  This  allowed  a reduction  in  the  number  of  simplex 
iterations  that  were  necessary  for  solving  the  problems  (see  exhibit  2 ). 

3)  In  each  cycle  of  the  solution  procedure,  a lower  limit  (*=  objective 
function  value  of  the  master  problem)  and  an  upper  limit  (calculated 
as  described  in  Lasdon  [3])  for  the  optimal  value  of  the  problem's 
objective  function  were  computed.  A solution  procedure  was  ended  when 

(upper  limit  - lower  limit) *100  ^ q 01 

upper  limit 

(Exhibit  3 shows  an  example  for  the  development  of  the  upper  and  lower 
limit  for  the  objective  function  value  of  the  optimal  solution  of  the 
problem  during  the  iterative  solution  procedure.) 
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(1)  lower  bound 

(2)  upper  bound 


5 10  15  20  25  SO  35 

cycle  of  the  solution  procedure 


Exhibit  3:  Upper  and  lower  bound  for  the  optimal  value  of  the  objec- 
tive function  in  different  cycles  of  the  solution  proce- 
dure (example:  experiment  I). 


7 


4.  The  experiments 


4.1  Optimization  problem 


The  experiments  were  based  on  an  optimization  problem  where  the 
block-angular  structure  of  the  matrix  was  formed  by  n=32  diagonal  subma- 
trices A (i=l,..,n)  and  a "main  matrix"  H * [B.  B,  ...  B D]  and  which 
i l JL  n 

could  be  formulated  mathematically  as 


max  z 


C1X1  + C2X2  + 


. . . + c x + dy 
n n 


s.t. 


Alxl 


A2X2 


£ 


a 

a 


1 

2 


A x 
r.  n 


B.x,  + B„x„  + ...  + B x +Dy=b 
11  2 2 n n J 


x1,y  ^ 0 (i»l,..,n) 


with  x^(l»l,..,n)  and  y representing  vectors  of  decision  variables. 

In  the  following,  we  will  refer  to  the  constraints  that  are  connec- 
ted with  the  main  matrix  H as  "common"  constraints.  The  size  and  density 
of  the  different  matrices  are  outlined  in  exhibit  4. 


Matrix  size  (rows, columns) 

(907,1265) 

Density 

0.71 

Number  of  submatrices 

32 

Size  (rows, columns) 

(28,39) 

Density 

18.58 

Size  of  the  main  matrix 

(11,1265) 

Density 

7.73 

Exhibit  4 
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The  experiments  were  aimed  at  testing  the  efficiency  cf  the  compu- 
tational concept  and  investigating  ways  of  impro-  Ing  the  convergence  of  the 
iterative  solution  process  by 


a)  an  appropriate  choice  of  the  feasible  starting  solution  that  has  to  be 
formulated  at  the  beginning  of  the  solution  process  (see  experiments  1, 
II,  III  and  VII), 

b)  the  incorporation  of  submatrices  into  the  master  problem  (see  experi- 


ments I,  IV  and  V)  and 

c)  the  variation  of  the  size  of  distinguished  subproblems  (see  experiments 
VI,  VII  and  X). 


The  organization  of  the  different  experiments  is  outlined  in  detail 
in  exhibit  5 . 

The  distinguished  starting  solutions  A,  B,  C and  D (see  column  2 of 
the  exhibit)  were  feasible  solutions  of  the  optimization  problem  that 
differed 

a)  in  the  values  of  their  objective  functions  (except  solutions  A and  D), 

b)  in  the  values  of  the  dual  variables  that  corresponded  to  the  common 
constraints  (except  solutions  A and  B)  and 

c)  in  the  values  of  the  primal  variables. 

The  number  of  constraints  in  the  master  problem  (see  column  6)  refers 
to  the  common  constraints  of  the  optimization  problem,  the  additional  con- 
straints necessary  for  forming  convex  combinations  of  subproblem  proposals 
and  (in  experiment  IV  and  V)  constraints  that  are  due  to  the  incorporation 
of  submatrices  into  the  master  problem. 
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Experiment  Identifica-  Deviation  of  Number  of  Matrix  size  Number  of  Main  aspect  o 

tion  of  the  objective  subproblems  of  subproblem  constraints  investigation 

starting  function  (m,n)  in  the  master 


In  addition  to  these  basic  experiments  some  computations  were 


performed  in  which 

a)  the  objective  function  of  the  optimization  problem  was  changed  in 
one  of  the  experiments  after  an  optimal  solution  with  regard  to  the 
original  objective  function  had  been  reached  (see  experiment  IX  in 
chapter  4.36)  and 

b)  differences  in  common  constraints  that  had  been  observed  in  the  basic 
experiments  were  examined  with  regard  to  their  influence  on  the  opti- 
mization process  (see  experiments  XI,  XII,  XIII  and  XIV  in  appendix  2). 

4.3  Discussion  of  results 

We  will  discuss  the  results  of  the  experiments  with  regard  to  the 
usefulness  of  an  approach  in  terms  of  its  effects  on 

a)  the  computation  time  and 

b)  the  generation  of  "good"  suboptimal  solutions  at  an  early  stage  of  the 
iterative  solution  procedure. 

In  the  following  context,  a suboptimal  solution  is  considered  to  be 
"good"  if  it  is  a sufficiently  good  approach  to  the  optimal  solution,  i.e., 
an  approach  that  would  allow  the  termination  of  the  iterative  solution 
procedure  if  some  deviation  between  the  computed  solution  and  the  optimal 
solution  is  considered  to  be  acceptable. 

4.31  Relevance  of  results 

The  results  that  are  of  relevance  for  an  experiment's  computation 
time  are  outlined  in  exhibits  6 and  7 . 

|. 

f 
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Number  of  Number  of  simplex  iterations  Number  of  simplex  Calculations  of  the 

generated  in  the  subproblems  (average)  iterations  in  the  type  a»b+c*d 

subproblem  master  problem  (in  1000) 


Average  core  time  (min:sec)  Average  transfer  time  (min: 

sub-  I per  master!  per  cycle  per  sub-  I per  master  | per 


Exhibit  7 : Some  results  of  the  experiments. 


In  exhibit  7,  the  terms  "core  time"  and  "transfer  time"  are  used. 

In  this  context,  the  term  "core  time"  refers  to  the  time  that  was  necessary 


for  performing  the  simplex  iterations  in  the  subproblems  and  the  master 
problems  and  for  performing  the  calculations  of  type  a"b+c*d,  the  calcula- 
tion of  indices,  etc.  in  the  FORTRAN  programs.  An  identification  of  the 
relevant  CPU  time  was  not  possible.  The  term  "transfer  time"  refers  mainly 
to  the  time  that  was  necessary  for 

a)  exchanging  data  between  LP  programs  and  FORTRAN  programs  (with  discs 
as  temporary  storage  units)  and 

b)  the  transfer  of  subproblems  and  master  problems  from  the  central  pro- 
cessor unit  to  storage  units  (discs)  and  reverse. 

At  any  time,  there  was  only  one  optimization  problem  (a  subproblem 
or  a master  problem)  in  the  central  processor  unit  for  performing  computa- 
tions. 

The  specified  average  core  and  transfer  times  are  closely  related 
to  the  used  computer  facilities  and  the  organization  of  the  programming 
cycle  whereas  the  other  figures  in  the  exhibits  are  related  to  the  optimi- 
zation problem  that  had  to  be  solved. 

It  should  be  noted  that  the  percentage  of  computation  time  that 
could  be  specified  as  core  time  was  relatively  low  in  all  experiments 
(between  11%  and  39%),  i.e.,  an  efficient  organization  of  the  storage  fa- 
cilities is  one  of  the  most  important  possibilities  for  increasing  the 
efficiency  of  the  optimization  process.  This  is  especially  true  with  re- 
gard to  the  LP  programs  which  were  responsible  for  most  of  the  needed 
computation  time  (usually  for  more  than  75%). 


Of  a more  general  Interest  are  the  variations  of  results  within  the 
distinguished  groups  of  experiments.  Their  analysis  can  provide  information 
which  could  be  of  interest  for  the  optimization  of  problems  with  a similar 
structure  in  the  problem  matrix.  The  same  is  true  for  a comparative  evalua- 
tion of  the  suboptlmal  solutions  that  were  generated  during  the  iterative 
solution  procedures. 

The  more  important  observations  and  conclusions  are  discussed  in 
detail  in  the  following  sections. 
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4.32  Different  starting  solutions 


A comparison  of  different  starting  solutions  was  based 

a)  on  differences  in  their  objective  fuction  value, 

b)  on  deviations  of  the  values  of  the  dual  variables  that  refer  to  the 
common  constraints  from  their  values  in  the  optimal  solution  and 

c)  on  deviations  of  the  values  of  primal  variables  from  their  values  in 
the  optimal  solution. 

A comparison  of  the  experiments  I,  II,  III,  and  VII  showed  that  it  is 
not  possible  to  judge  the  usefulness  of  a specific  starting  solution  merely 
on  the  basis  of  one  of  these  criteria,  not  to  mention  the  problem  of  deter- 
mining at  least  some  rough  estimates  for  the  optimal  values  of  the  objective 
function  or  the  primal  and  dual  variables. 

For  example,  the  experiments  I,  II  and  III  could  be  finished  in  an 
approximately  identical  number  of  cycles  despite  the  fact  that  the  objective 
function  values  of  the  starting  solutions  differed  significantly  (see  ex- 
hibit 6). 

Furthermore,  differences  in  the  objective  function  values  of  differ- 
ent starting  solutions  were  of  no  relevance  for  the  development  of  these 
values  during  the  solution  procedure  (see  exhibits  8 and  9 ) . 

The  same  conclusion  could  be  drawn  with  regard  to  the  values  of  the 
dual  variables  that  correspond  to  the  common  constraints  and  with  regard  to 
the  values  of  the  primal  variables .which  were  examined  by  distinguishing 

a)  variables  with  coefficients  in  the  main  matrix  only, 

b)  variables  with  coefficients  in  the  submatrices  only  and 

c)  other  variables. 
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objective  func- 
tion value 
( million) 


* r 


(1)  experiment  I 

(2)  experiment  II 

(3)  experiment  III 


.Q / 


cycles 


Exhibit  10  shows  the  relative  deviations  of  the  dual  variables' 


values  in  the  different  starting  solutions  from  their  optimal  values. 
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Exhibit  10 


Comparing  them  with  the  number  of  cycles  that  were  necessary 
for  performing  the  experiments  (see  especially  experiments  I and  III 
with  different  deviations  but  identical  numbers  of  cycles)  and  the  devel- 
opment of  the  objective  function  values  during  the  solution  procedures 
(see  especially  experiments  I and  II  with  identical  deviations  but  differ- 
ent developments  of  the  objective  function  values  as  shown  in  exhibit  8 ) 
no  definite  relationships  could  be  identified. 

The  same  is  true  for  the  development  of  the  relative  deviations 
during  the  solution  procedures  as  can  be  concluded  from  exhibit  11.  It 
•hows  for  each  of  the  common  constraints  the  maximal  relative  deviations 
of  the  dual  variables'  values  from  their  optimal  values  that  could  be  ob- 
•*rvsd  in  the  experiments  (a)  during  the  whole  solution  procedure 
(column  ''0''),  (b)  after  10  cycles  (column  ”10"),  (c)  after  20  cycles 
(column  ”20")  and  (d)  after  30  cycles  (column  ”30"). 
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(1)  experiment  I 

(2)  experiment  II 

(3)  experiment  III 

(4)  experiment  VII 

(5)  experiment  VIII 


Exhibit  12:  Development  of  the  objective  function  value  during  experi- 
ment VIII  (for  a comparison  see  experiments  I,  II,  III  and  VII) 
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However,  the  determination  of  a "useful"  starting  solution  could 
be  simplified  by  specifying  not  a single  solution  but  several  ones,  each 
of  which  might  be  less  useful  or  even  not  feasible  but  which  could  be  com- 
bined to  a more  useful  "starting  solution"  in  the  first  master  problem  of 
the  solution  procedure  (see  experiment  VIII  and  exhibit  12  ) , 

4.33  Incorporation  of  submatrices  into  the  master  problem 

The  incorporation  of  submatrices  into  the  master  problem  (while  re- 
ducing the  number  of  subproblems)  results 

a)  in  a higher  flexibility  of  the  master  problem  and 

b)  usually  in  a change  in  the  time  that  is  required  for  computing  one  cycle 
of  the  solution  procedure. 

Because  of  the  master  problem's  higher  flexibility  in  the  experiments 
IV  and  V compared  to  experiment  I it  was  possible  to  reach  in  the  first  iter- 
ation objective  function  values  that  differed  from  the  optimal  value  not 
more  than  16  Z or  14  Z,  respectively.  In  addition,  the  number  of  cycles 
could  be  reduced  from  42  to  15  or  13,  respectively.  (See  exhibit  13.) 

The  computation  time  that  is  required  for  computing  one  cycle  of  the 
solution  procedure  is  subject  to  conflicting  influences.  On  the  one  side, 
it  tends  to  decrease  because  of  a reduction  in  the  number  of  subproblems, 
on  the  other  side,  it  tends  to  increase  because  of  the  enlargement  of  the 
master  problem.  The  resulting  changes  in  the  computation  time  depend  on  the 
computational  facilities  and  the  used  computer  programs  and  have,  therefore, 
to  be  estimated  individually. 

For  example,  a comparison  of  experiments  IV  and  V showed  that  in  this 
case  the  incorporation  of  an  additional  submatrix  into  the  master  problem 
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would  be  of  interest  if  the  number  of  cycles  could  be  reduced  by  (theoreti- 
cally) 0.1. 

If  such  a calculation  is  based  on  figures  that  have  been  obtained 
during  the  first  cycle  of  the  solution  procedure,  the  reduction  in  the 
number  of  cycles  that  is  considered  to  be  necessary  to  make  an  incorpora- 
tion of  submatrices  into  the  master  problem  interesting  usually  turns  out 
to  be  higher  than  if  based  on  average  figures.  (This  is  due  to  the  fact 
that  the  solution  of  subproblems  and  master  problems  usually  requires 
more  simplex  iterations  during  the  first  cycles  of  the  solution  procedure 
(see  also  exhibit  2 ) .) 

In  the  case  of  experiments  IV  and  V,  the  relevant  number  would  be 
0.5  cycles.  Therefore,  if  a calculation  based  on  these  figures  shows  an 
incorporation  of  submatrices  into  the  master  problem  to  be  efficient,  it 
can  usually  be  implied  that  it  will  be  efficient  with  regard  to  the  (still 
unknown)  average  figures  as  well. 

4.34  Variations  in  the  number  and  size  of  subproblems 

Changes  in  the  number  and  size  of  distinguished  subproblems  in- 
fluence (a)  the  number  of  cycles  that  are  necessary  for  solving  an  opti- 
mization problem  and  (b)  the  time  required  for  performing  a single  cycle 
within  the  solution  procedure. 

Theoretically,  the  number  of  necessary  cycles  can  be  minimized  by 
distinguishing  as  many  subproblems  as  possible  (see,  for  example,  Lasdon  [3]). 

As  an  example,  the  number  of  cycles  that  were  necessary  for  per- 
forming the  experiments  X,  VII  and  VI  could  be  reduced  from  103  to  49  and  35. 


In  addition,  by  distinguishing  more  but  smaller  subproblems  it 
was  possible  to  reduce  the  average  time  that  was  necessary  for  performing 
the  simplex  iterations  in  a cycle  of  the  procedure.  This  was  due  to  the 
fact  that  the  computation  of  a simplex  iteration  takes  less  time  in  smaller 
problems  and  that  this  reduction  in  computation  time  was  not  compensated 
in  the  experiments  by  an  observed  slight  Increase  in  the  number  of  sim- 
plex iterations. 

However,  due  to  the  use  of  standard  LP  programs,  the  increase  in 
the  number  of  distinguished  subproblems  is  only  useful  within  a certain 
range.  The  activation  of  these  programs  requires  a basic  amount  of  time 
which  is  independent  of  the  size  of  an  optimization  problem.  Because  of 
this,  the  average  time  for  computing  a cycle  in  experiment  VI  was  about 
500  seconds  longer  than  in  experiment  VII,  while  the  transition  from 
experiment  X to  experiment  VII  still  caused  a reduction  in  computation 
time  of  about  250  seconds. 

The  extent  to  which  an  increase  in  the  number  of  distinguished 
subproblems  remains  useful  can  only  be  estimated  individually.  If  such 
an  estimation  is  based  on  figures  that  have  been  obtained  during  one  of 
the  f irst  cycles  of  the  solution  procedure, it  has  to  be  taken  into  account 
that  the  solving  of  subproblems  usually  requires  more  simplex  iterations 
at  the  beginning  than  towards  the  end  of  the  solution  procedure,  i.e., 
the  possible  savings  in  computation  time  are  usually  lower  than  indicated 
during  the  first  cycles  of  the  procedure. 


4.35  General  remarks  about  the  values  of  primal  and  dual  variables 


In  situations  where  the  solution  process  has  to  be  ended  before  an 
optimal  solution  has  been  reached,  the  values  of  the  primal  and  dual  vari- 
ables and  their  deviations  from  the  optimal  values  might  be  of  interest. 

In  the  experiments,  the  values  of  the  primal  variables  usually  varied 
only  within  a relatively  small  range  within  the  feasible  range  during  the 
iterative  process  (disregarding  the  very  first  cycles).  However,  the  devia- 
tions from  their  optimal  values  could  still  be  relatively  high  (with  regard 
to  the  value  of  the  objective  function)  in  almost  optimal  solutions  (see 
exhibit  14).  There  were  no  significant  differences  between  variables  with 

a)  coefficients  only  in  the  common  constraints, 

b)  coefficients  only  in  the  submatrices  and 

c)  other  variables. 

The  values  of  the  dual  variables  (that  corresponded  to  the  common 
constraints)  that  had  been  observed  during  the  iterative  solution  processes 
are  outlined  in  appendix  4.  They  varied  significantly  around  their  opti- 
mal values  and  usually  showed  some  kind  of  convergence  properties, as  demon- 
strated in  exhibit  15  (see  also  exhibit  11) . 

4.36  Postoptlmal  calculations 

For  the  examination  of  the  effects  of  changes  in  the  objective 
function  or  the  capacity  vector  of  an  optimization  problem  on  the  computed 
optimal  solution,  the  iterative  solution  procedure  has  to  be  restarted. 

r E 

However,  the  columns  of  the  last  cycle's  master  problem  can  be  in- 
cluded in  the  master  problem  of  the  restarted  process  if  changes  in  the 
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number  of 
units 


Exhibit  14:  Changes  in  the  value  of  a primal  variable  during 
experiment  I. 


objective  function  or  the  part  of  the  capacity  vector  that  refers  to  the 
common  constraints  have  to  be  considered.  While  the  columns  can  be  used 
unchanged  in  cases  of  changes  in  the  capacity  vector,  their  coefficients 
in  the  objective  function  have  to  be  revised  in  cases  where  the  objective 
function  had  been  changed. 

An  example  for  such  a computation  is  shown  in  exhibit  16,  where 
experiment  II  was  continued  after  39  cycles  with  a revised  objective 
function  (experiment  IX). 
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Exhibit  16:  Solutions  of  subproblems  that  were  Included  in  the  optimal 

basis  solution  of  the  master  problem  and  the  relative  devia- 
tions of  the  objective  functions’  values  from  their  optimal 
values  when  objective  function  1 was  replaced  by  objective 
function  2 (experiment  IX). 
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5.  Conclusion 

The  solution  of  large  optimization  problems  with  a decomposition 
program  that  is  based  on  efficient  standard  LP  programs  is  usually  no 
alternative  to  a simultaneous  optimization,  i.e.  an  optimization  with- 
out decomposition  by  means  of  a standard  LP  program.  The  realization  of 
the  computational  concept  requires  additional  efforts  for  establishing 
the  programming  cycle,  organizing  the  data  etc., which  reduces  its 
efficiency. 

However,  for  the  solution  of  optimization  problems  that  are  too 
large  for  the  available  computational  facilities, the  proposed  procedure 
might  be  efficient. 

As  the  experiments  showed,  an  appropriate  identification  of  sub- 
problems could  allow  the  reduction  of  core  time  that  is  necessary  for 
solving  an  optimization  problem  to  an  amount  similar  to  the  amount  of 
time  that  is  necessary  for  a simultaneous  optimization  (see,  for  example, 
exhibit  5). 
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problems 

size  of  optimization  problems  that  can  be  solved  by  means  of 
standard  LP  programs  is  limited  by 

a)  the  capacity  of  the  available  computers  and 

b)  the  efficiency  of  the  standard  LP  programs. 

The  crucial  figure  in  this  context  is  the  number  of  constraints  that  have 
been  formulated  in  the  problem  (see  exhibit  17  ) . 


Name  of  the  program  (Company) 

Rows  (m) 

Columns  (n) 

FMPS  (UNIVAC) 

8 192 

unlimited 

ILONA  (UNIVAC) 

m+n  * 

9 500 

LP  4004  (SIEMENS)* 

6 000 

32  767 

LP  600/6000  (HONEYWELL  BULL)* 

4 095 

262  000 

MARIE  CLAIRE  (UNIVAC) 

12  000 

unlimited 

MPSX  (IBM) 

16  383 

unlimited 

0PHELIE  II  (CONTROL  DATA)* 

10  000 

unlimited 

OPTIMA  (CONTROL  DATA)* 

4 094 

unlimited 

XDLA  (ICL) 

25  000** 

100  000 

* Information  from  1972,  others 

1973  (XDLA  1974)  1 

**  Problems  with  m=2500  reported 

Exhibit  17 


&ES2&. 


The 


For  problems  with  a block-angular  structure  in  the  matrix  the  stan- 
dard LP  programs  "LP  600/6000"  and  "Optima"  offer  an  alternative  solution 
procedure  which  is  based  on  a decomposition  method  developed  by  Orchard- 
Hays  [ 4 ] and  which  is  designed  for  solving  problems  with  up  to  50,000 
constraints.  However,  practical  experiences  with  this  procedure  seem  to 

. D 

be  very  scarce. 

1)  Orchard-Hays  in  private  correspondence. 
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Aggend 1x^2 : Additional  experiments  with  different  common  constraints 

In  the  experiments  I - X,  it  turned  out  that  some  of  the  dual 
variables  that  corresponded  to  the  common  constraints  always  reached 
their  optimal  values  in  the  first  cycle  of  the  solution  procedure  while 
the  remaining  dual  variables  changed  their  values  iteratively.  The 
common  constraints  that  corresponded  to  the  first  group  of  dual  variables 
were  numbered  8-11,  the  remaining  commom  constraints  1-7. 

For  getting  more  information  about  the  influence  of  different 
kinds  of  common  constraints  on  the  solution  procedure,  experiments 
XI,  XII,  XIII  and  XIV  were  outlined , basically  as  described  for  experiment  X 
but  with  the  following  specifications: 

1)  In  experiments  XI  and  XII,  the  master  problems  included  only  one  of  the 
common  constraints  whereas  the  remaining  common  constraints  were  added 
to  the  subproblems.  In  experiment  XI  we  considered  explicitly  constraint 
11,  in  experiment  XII  constraint  1. 

2)  In  experiments  XIII  and  XIV,  the  master  problems  included  only  one 

of  the  distinguished  groups  of  common  constraints  whereas  the  remaining 
group  of  common  constraints  was  added  to  the  subproblems.  In  experi- 
ment XIII  we  considered  explicitly  the  common  constraints  8-11  in  the 
master  problems,  in  experiment  XIV  the  common  constraints  1-7. 

The  results  of  the  computations  are  outlined  in  exhibits  18  to  19. 

The  most  noteworthy  results  seem  to  be: 

a)  the  similarity  in  the  experiments  XI  and  XII  despite  the  fact  that  differ- 
ent kinds  of  common  constraints  were  considered, 

b)  the  solution  procedure  of  experiment  XIII. 
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Cycle 


Lower 

bound 


Upper  bound 
computed  in 
cycle 


"Lowest"  com- 
puted upper 
bound 


Experiment  XI 


Experiment  XIV 


♦ 955  266 

6 140  388 
6 181  931 
6 193  908 
6 194  023 
6 194  083 


6 268  201 
17  117  741 
6 461  359 

6 246  256 
6 194  531 
6 194  097 


6 268  201 


6 248  256 
6 194  531 
6 1 94  097 


1 

2 

4 955  266 
• 

108  019  791 

108  019  791 

3 

• 

12  857  900 

12  857  900 

4 

• 

15  858  291 

• 

5 

13  687  413 

• 

6 

• 

16  955  389 

• 

7 

• 

8 986  892 

8 986  892 

8 

• 

12  328  003 

• 

9 

• 

9 698  789 

• 

10 

• 

9 230  287 

• 

11 

• 

8 923  979 

8 923  979 

12 

• 

9 760  913 

• 

13 

• 

8 559  648 

8 559  648 

14 

• 

7 945  981 

7 945  981 

15 

• 

10  879  345 

• 

16 

• 

12  621  243 

• 

17 

• 

8 623  734 

• 

18 

• 

8 204  9 22 

• 

19 

5 335  049 

7 142  742 

7 142  742 

20 

5 425  695 

14  211  884 

• 

21 

5 674  791 

9 989  577 

• 

22 

5 678  287 

7 737  021 

• 

experiment  ended 


Exhibit  18:  Development  of  the  lower  and  upper  bound  for  the  optimal 

value  of  the  objective  function  in  experiments  XI,  XII,  XIII 
and  XIV.  (Optimal  value:  6,194,083). 
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Cycle 


12  3456789  10 


Experiment  XI: 
(common 

constraint  11) 


0 -1073.6  -101.0  -68.2  -58.7  -58.6  -58.6 


Experiment  XII: 
(common 
constraint  1) 


0 100.1  311.3  159.3  225.3  193.1  209.9  218.3  214.4  215.2 


Experiment  XIII: 
(common  con- 
straints 8-11) 


optimal  values  in 

first  cycle  (remained  unchanged) 


Exhibit  19:  Values  of  the  dual  variables  that  corresponded  to  the  common 
constraints  in  experiments  XI  (common  constraint  11),  experi- 
ment XII  (common  constraint  1)  and  experiment  XIII  (common 
constraints  8-11)  during  the  solution  procedure. 


The  exhibits  in  this  section  show  for  the  basic  experiments  the 
development  of 

a)  a lower  bound  for  the  optimal  objective  function  value  which  equals 


the  objective  function  value  of  a cycle's  master  problem  (in  the 
exhibits  referred  to  as  "lower  bound"), 
b)  an  upper  bound  for  the  optimal  objective  function  value  which  was 
computed  in  each  cycle  in  the  way  suggested  by  Lasdon  [ 3]  (in  the 
exhibits  referred  to  as  "upper  bound  in  cycle")  and 


c)  the  lowest  upper  bound  for  the  optimal  objective  function  value  that 
had  been  computed  in  preceding  cycles  (in  the  exhibits  referred  to 
as  "upper  bound"). 
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Experiment 


Append ix_4:  Values  of  dual  variables  in  the  basic  experiments 


The  exhibits  In  this  section  show  the  development  of  the  values 

i 

of  the  dual  variables  that  corresponded  to  the  common  constraints  during 
the  solution  procedures  of  the  basic  experiments  (only  common  constraints 
1-7  considered,  as  the  dual  variables  that  referred  to  constraints  8-11 
remained  unchanged) . 
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Experiment  I 


Cycle  Common  constraint  Cycle 
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